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Background/context: 

Description of prior research, its intellectual context and its policy context. 



Without defining and assessing the degree to which an intervention was implemented as 
intended (i.e., implementation fidelity), claims of treatment effectiveness may be unjustified. 
Despite the value of measuring implementation fidelity, recent reviews of studies in K-12 school 
settings have illustrated that many inconsistencies and omissions in measuring fidelity exist 
(Dusenbury, Brannigan, Falco, & Hansen, 2003; O’Donnell, 2008). 

No review of preschool studies and the ways in which they assess fidelity of 
implementation has yet been published. Therefore, it is difficult to determine if the problems 
inherent in K-12 studies are also evident in preschool interventions. The movement to establish 
effective preschool programs has gained momentum. The fact that some children are not 
prepared to enter kindergarten is a source of great concern for educational researchers, 
practitioners, and policy makers. This concern prompted the U.S. Department of Education to 
invest millions of dollars in grants for evaluations of preschool curricula (Barnett, 2008). With 
increased funding for such studies and heightened attention to the importance of implementation 
fidelity comes the emergence of a variety of fidelity instruments. However, the increase in 
number of fidelity instruments does not guarantee that these measures effectively assess 
implementation. 



Purpose / objective / research question / focus of study: 

Description of what the research focused on and why. 



This paper examines measures used by studies associated with Preschool Curriculum 
Evaluation Research Initiative (PCER) funded by the Institute of Education Sciences. Analysis 
of 17 measures of fidelity used by 13 curriculum interventions highlights how research in 
preschools is and is not measuring fidelity of implementation. The following research questions 
are pursued: 

1. In what ways do the measures used in these studies represent the three primary elements 
of implementation: (a) adherence, (b) exposure, and (c) participant responsiveness? 

2. Do studies effectively use measures to differentiate between teachers’ fidelity to crucial 
elements of the curriculum and teachers’ general instructional quality (i.e., high fidelity 
as compared to good teaching)? 

Population / Participants / Subjects: 

Description of participants in the study: who (or what ) how many, key features (or characteristics). 



Fidelity measures were gathered from 13 research studies funded by the Institute of 
Education Sciences’ Preschool Curriculum Evaluation Research Initiative (PCER) and located in 
Tennessee, North Carolina/Georgia, New Hampshire, Florida/Kansas/New Jersey, Texas, 
Virginia, California/New York, Wisconsin, Missouri, New Jersey, and Florida. This paper 
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evaluates instruments representing 12 separate curricula implemented within these 13 
interventions. 



Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 



Seventeen fidelity instruments were coded on the instrument and item level (see Table 1 
in Appendix B for a list of fidelity measures used by each study). At the instrument level, each 
measure was coded for the inclusion of teacher interview, guidance on optimal length of 
observation, and opportunities for observers to document the length of observed activities. At 
the item level, the coding schema identified individual items that included direct reference to the 
targeted curriculum and items that provided some means to assess the quality of a particular 
feature of the environment or instruction. Additionally, each item was then designated as relating 
to either structural elements of the classroom setting or instructional features associated with 
teacher or child behaviors. 

Findings / Results: 

Description of main findings with specific details. 



Results show that few studies use measures that comprehensively represent the three 
primary elements of implementation. Only 5.7% of all items across instruments (N = 1,113 
items) and only 5 of the 16 measures contained any assessment of adherence. Likewise, less than 
half of the studies provided measures representing children’s exposure to the curriculum. 
Directions included in the measures often lacked guidance on the ideal length of the observation 
and frequently omitted space for observers to indicate the total length of observation. Only 9 of 
the 15 measures analyzed at the item level included scheduling items. Thus, few measures 
evaluated how well teachers followed guidance on the intended length and sequence of curricular 
activities. Additionally, only 3 of the 16 fidelity measures included representation of teacher 
responsiveness in the form of a teacher interview. The included fidelity measures virtually 
ignored children’s response to structural and instructional elements of the interventions. 

Conclusions: 

Description of conclusions and recommendations based on findings and overall study. 



Overall, analysis of these 16 fidelity measures reveals that the corresponding preschool 
studies insufficiently measured the primary components of fidelity. In many cases, the measures 
used in these studies were more effective at assessing general quality of instruction and less 
successful at evaluating fidelity of implementation. Despite increased attention, measurement of 
implementation fidelity in preschool research continues to be challenging and development of 
measures that more comprehensively represent implementation fidelity more is essential. 
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Appendices 

Not included in page count. 
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Appendix B. Tables and Figures 

Not included in page count. 

Table 1 

Fidelity Measures Used by Study 



Study Location 


Curriculum 


Fidelity Measure 


NH 


Creative Curriculum with Ladders to 
Literacy 


1 . Ladder to Literacy Implementation Checklist 
(Classroom Activities) 

2. Ladder to Literacy Implementation Checklist 
(Scaffolding) 


NY&CA 


Pre-K Mathematics + DLM Express 


1. Early Mathematics Classroom Observation 

2. Fidelity of Implementation Record Sheet 

3. Classroom Obs of Early Math Environment and 
Teaching (COEMET) 


NC, GA & NH 


Creative Curriculum 


1 . Creative Curriculum Checklist 


TX 


Doors to Discovery 


1. Doors to Discovery: Curriculum Fidelity Checklist 

2. PCER Teacher Checklist* 


TX 


Let's Begin with Letter People 


1. Let's Begin: Curriculum Fidelity Checklist 

2. PCER Teacher Checklist* 


WI 


Project Approach 


1. The Project Approach Fidelity Scale 


MO 


Project Construct 


1. Project Construct: Early Childhood Observation Sr 
(PC-ECCOS) 


VA 


Language Focused Curriculum (LFC) 


1 . LFC Fidelity Checklist 


FL 


DLM Early Childhood Express with 
Open Court Pre-K 


1. DLM/Open Court Classroom Observation 


FL 


Literacy Express 


1. Literacy Express Classroom Observation 


NJ 


Ready, Set, Leap! 


1. Fidelity Observation 
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2. Classroom Lang Arts Systematic Sampling and 
Instructional Coding (CLASSIC) 



TN Bright Beginnings 1. Combined CC and BB checklist 

TN Creative Curriculum 1. Combined CC and BB checklist 

* PCER Checklist is the same for both TX studies (Doors to Discovery and Let’s Begin with Letter People 
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